Human-computer interaction for virtual-real fusion

面向虚实融合的人机交互涉及计算机科学、认知心理学、人机工程学、多媒体技术和虚拟现实等领域，旨在提高人机交互的效率，同时响应人类认知与情感的需求，在办公教育、机器人和虚拟/增强现实设备中都有广泛应用。本文从人机交互涉及感知计算、人与机器人交互及协同、个性化人机对话和数据可视化等4个维度系统阐述面向虚实融合人机交互的发展现状。对国内外研究现状进行对比，展望未来的发展趋势。本文认为兼具可迁移与个性化的感知计算、具备用户行为深度理解的人机协同、用户自适应的对话系统等是本领域的重要研究方向。;Virtual-real human-computer interaction(VR-HCI) is an interdisciplinary field that encompasses human and computer interactions to address human-related cognitive emotional needs. This knowledge integrates domains such as science, psychology, ergonomics, multimedia technology, virtual reality. With the advancement of big data artificial intelligence, VR-HCI benefits industries like education, healthcare, robotics, entertainment, increasingly recognized a key supporting technology for metaverse-related development. In recent years, machine learning-based analysis has evolved, particularly in applications robotics wearable interaction devices. As result, focused on challenging issue creating intelligent anthropomorphic systems. literature review examines growth from four aspects:perceptual computing, human-machine coordination, dialogue interaction, visualization. Perceptual computing aims model daily life behavior, processes, contexts personalized efficient interactions. discussion covers three perceptual aspects related pathways, objects, scenes. Human-machine scenarios involve real-world integration which are divided into primary perception types:visual-based, sensor-based, wireless non-contact. Object-based subdivided personal group contexts, while scene-based physical behavior contexts. primarily technical disciplines mechanical electrical engineering, control other arts or humanistic psychology design. Human-robot can be categorized by functional mechanisms 1) collaborative operation robots, 2) service assistance 3) social, educational robots. Key modules systems include speech recognition, speaker system, synthesis. The level intelligence these further enhanced considering users' inherent characteristics, pronunciation, preferences, emotions, attributes. For it mainly involves relevant well Humans-robots segmented categories terms its mechanism:1) entertainment system consists microphone sensor pick up signal, then converted text information through recognition module. process information, understand user's intention, generates reply. Finally, speech-synthesized module convert reply completing process. improved combining characteristics optimizing various system. transformation visualization, benched performing cleaning tasks tabular data, tools R Python perform well. Many software have developed graphical user interfaces assist users tasks, Microsoft Excel, Tableau Prep Builder, OpenRefine. Current recommendation-based algorithms interactive beneficial transform easily. Researchers also network structures. We analyze transformation, table visual comparison, 4) code visualization identify several future research directions VR-HCI, namely designing generalized building cooperation with deep understanding expanding user-adaptive still lacks joint multiple devices individual differences perception. Furthermore, most use models, neglecting differences, resulting lower accuracy, making difficult apply actual settings. Therefore, trends required multimodal, transferable, personalized, scalable research. systematic approach necessary constructing design collaboration. requires in-depth understanding, construction datasets, long-term experience. current mostly focuses open-domain systems, pre-trained models improve modeling accuracy intentions, knowledge. Future should aimed at developing more conversations cater HCI, composed two parts:1) workers aspects, e. g., appropriate types recommendations consistent real-time support massive data. focus among users, including mechanisms, resolving conflicts operation, visualizing complex codes, evaluating effectiveness methods, recording displaying behaviors. summary, development provide new opportunities challenges towards Metaverse, potential seamlessly integrate real worlds.